GAO 


United  States  Government  Accountability  Office _ 

Report  to  Congressional  Committees 


F-35  JOINT  STRIKE 
FIGHTER 

Problems  Completing 
Software  Testing  May 
Hinder  Delivery  of 
Expected  Warfighting 
Capabilities 


GAO-14-322 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

MAR  2014  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2014  to  00-00-2014 

4.  TITLE  AND  SUBTITLE 

F-35  joint  Strike  Fighter:  Problems  Completing  Software  Testing  May 
Hinder  Delivery  of  Expected  Warfighting  Capabilities 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Government  Accountability  Office, 441  G  Street 

NW, Washington, DC, 20548 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

41 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


March  2014 


Gm 

Highlights 

Highlights  of  GAO-14-322,  a  report  to 
congressional  committees 


F-35  JOINT  STRIKE  FIGHTER 

Problems  Completing  Software  Testing  May  Hinder 
Delivery  of  Expected  Warfighting  Capabilities 


Why  GAO  Did  This  Study 

The  F-35  Lightning  II,  also  known  as 
the  Joint  Strike  Fighter,  is  DOD’s  most 
costly  and  ambitious  acquisition 
program.  The  program  seeks  to 
develop  and  field  three  aircraft  variants 
for  the  Air  Force,  Navy,  and  Marine 
Corps  and  eight  international  partners. 
The  F-35  is  integral  to  U.S.  and 
international  plans  to  replace  existing 
fighter  aircraft  and  support  future 
combat  operations.  Total  U.S.  planned 
investment  in  the  F-35  program  is 
approaching  $400  billion  to  develop 
and  acquire  2,457  aircraft  through 
2037,  plus  hundreds  of  billions  of 
dollars  in  long-term  spending  to 
operate  and  maintain  the  aircraft. 

The  National  Defense  Authorization 
Act  for  Fiscal  Year  2010  mandated  that 
GAO  review  the  F-35  acquisition 
program  annually  for  6  years.  In  this, 
GAO’s  fifth  annual  report  on  the  F-35, 
GAO  assesses  the  program’s 

(1)  ongoing  development  and  testing, 

(2)  long-term  affordability,  and 

(3)  manufacturing  progress. 

GAO  reviewed  and  analyzed 
manufacturing  data  through  December 
2013,  program  test  plans,  and  internal 
DOD  analyses,  and  spoke  with  DOD, 
program,  and  contractor  officials. 

What  GAO  Recommends 

GAO  recommends  that  DOD  assess 
and  identify  the  specific  capabilities 
that  realistically  can  be  delivered  to  the 
military  services  to  support  their 
respective  initial  operational 
capabilities,  and  share  its  findings  with 
the  Congress  and  military  services 
prior  to  July  2015.  DOD  concurred  with 
this  recommendation. 


View  GAO-14-322.  For  more  information, 
contact  Mike  Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov 


What  GAO  Found 

Delays  in  developmental  flight  testing  of  the  F-35’s  critical  software  may  hinder 
delivery  of  the  warfighting  capabilities  the  military  services  expect.  F-35 
developmental  flight  testing  comprises  two  key  areas:  mission  systems  and  flight 
sciences.  Mission  systems  testing  verifies  that  the  software-intensive  systems 
that  provide  critical  warfighting  capabilities  function  properly  and  meet 
requirements,  while  flight  sciences  testing  verifies  the  aircraft’s  basic  flying 
capabilities.  Challenges  in  development  and  testing  of  mission  systems  software 
continued  through  2013,  due  largely  to  delays  in  software  delivery,  limited 
capability  in  the  software  when  delivered,  and  the  need  to  fix  problems  and  retest 
multiple  software  versions.  The  Director  of  Operational  Test  and  Evaluation 
(DOT&E)  predicts  delivery  of  warfighting  capabilities  could  be  delayed  by  as 
much  as  13  months.  Delays  of  this  magnitude  will  likely  limit  the  warfighting 
capabilities  that  are  delivered  to  support  the  military  services’  initial  operational 
capabilities — the  first  of  which  is  scheduled  for  July  2015 — and  at  this  time  it  is 
not  clear  what  those  specific  capabilities  will  be  because  testing  is  still  ongoing. 

In  addition,  delays  could  increase  the  already  significant  concurrency  between 
testing  and  aircraft  procurement  and  result  in  additional  cost  growth.  Without  a 
clear  understanding  of  the  specific  capabilities  that  will  initially  be  delivered, 
Congress  and  the  military  services  may  not  be  able  to  make  fully  informed 
resource  allocation  decisions.  Flight  sciences  testing  has  seen  better  progress, 
as  the  F-35  program  has  been  able  to  accomplish  nearly  all  of  its  planned  test 
flights  and  test  points.  Testing  of  the  aircraft’s  operational  capabilities  in  a 
realistic  threat  environment  is  scheduled  to  begin  in  2015.  The  program  has 
continued  to  make  progress  in  addressing  some  key  technical  risks. 

To  execute  the  program  as  planned,  the  Department  of  Defense  (DOD)  will  have 
to  increase  funds  steeply  over  the  next  5  years  and  sustain  an  average  of  $12.6 
billion  per  year  through  2037;  for  several  years,  funding  requirements  will  peak  at 
around  $15  billion.  Annual  funding  of  this  magnitude  clearly  poses  long-term 
affordability  risks  given  the  current  fiscal  environment.  The  program  has  been 
directed  to  reduce  unit  costs  to  meet  established  affordability  targets  before  full- 
rate  production  begins  in  2019,  but  meeting  those  targets  will  be  challenging  as 
significant  cost  reductions  are  needed.  Additionally,  the  most  recent  cost 
estimate  for  operating  and  supporting  the  F-35  fleet  is  more  than  $1  trillion,  which 
DOD  officials  have  deemed  unaffordable.  This  estimate  reflects  assumptions 
about  key  cost  drivers  the  program  can  control,  like  aircraft  reliability,  and  those  it 
cannot  control,  including  fuel  costs,  labor  costs,  and  inflation  rates.  Reliability  is 
lower  than  expected  for  two  variants,  and  DOT&E  reports  that  the  F-35  program 
has  limited  additional  opportunities  to  improve  reliability. 

Aircraft  manufacturing  continued  to  improve  in  2013,  and  management  of  the 
supply  chain  is  evolving.  As  the  number  of  aircraft  in  production  has  increased, 
critical  learning  has  taken  place  and  manufacturing  efficiency  has  improved.  For 
example,  the  prime  contractor  has  seen  reductions  in  overall  labor  hours  needed 
to  manufacture  the  aircraft,  as  expected.  In  2013,  the  contractor  delivered  35 
aircraft  to  the  government,  5  more  than  it  delivered  in  2012  and  26  more  than  it 
delivered  in  2011.  The  prime  contractor  has  put  in  place  a  supplier  management 
system  to  oversee  key  supplier  performance. 
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March  24,  2014 
Congressional  Committees 

With  estimated  acquisition  costs  approaching  $400  billion,  the  F-35 
Lightning  II — also  known  as  the  Joint  Strike  Fighter — is  the  Department  of 
Defense’s  (DOD)  most  costly  and  ambitious  acquisition  program.  The 
program  is  developing  and  fielding  a  family  of  next  generation  fighter 
aircraft,  incorporating  low  observable  (stealth)  technologies  as  well  as 
advanced  sensors  and  computer  networking  capabilities  for  the  United 
States  Air  Force,  Navy,  and  Marine  Corps  as  well  as  eight  international 
partners.1  The  F-35  family  is  comprised  of  three  aircraft  variants:  (1)  a 
conventional  takeoff  and  landing  (CTOL)  variant,  (2)  a  short  takeoff  and 
vertical  landing  (STOVL)  variant,  and  (3)  a  carrier-suitable  variant  (CV). 
The  F-35  is  integral  to  U.S.  and  partner  plans  to  replace  existing  fighter 
aircraft  and  support  future  combat  operations.  According  to  current  plans, 
the  U.S.  portion  of  the  program  will  require  annual  acquisition  funding  of 
more  than  $12  billion  on  average  through  2037  to  complete  development 
and  procure  a  total  of  2,457  aircraft.  In  addition,  the  F-35  fleet  is 
estimated  to  cost  around  $1  trillion  to  operate  and  support  over  its 
lifetime.  In  a  time  of  austere  federal  budgets,  cost  projections  of  this 
magnitude  pose  significant  fiscal  challenges. 

We  have  reported  on  F-35  issues  for  many  years.2  Over  time  we  have 
reported  significant  cost,  schedule,  and  performance  problems  and  have 
found  that  those  problems,  in  large  part,  can  be  traced  to  (1)  decisions 
made  at  key  junctures  without  adequate  product  knowledge;  and  (2)  a 
highly  concurrent  acquisition  strategy  with  significant  overlap  among 
development,  testing,  and  manufacturing  activities.  We  have  made 
numerous  recommendations  aimed  at  addressing  these  issues,  and  DOD 
has  taken  action  to  address  them  to  varying  degrees.3  In  March  2012, 


^he  international  partners  are  the  United  Kingdom,  Italy,  the  Netherlands,  Turkey, 
Canada,  Australia,  Denmark,  and  Norway.  These  nations  contributed  funds  for  system 
development  and  signed  agreements  to  procure  aircraft.  In  addition,  Israel  and  Japan 
have  signed  on  as  foreign  military  sales  customers. 

2See  related  GAO  products  at  the  end  of  this  report. 

3See  appendix  I  for  a  matrix  of  prior  GAO  reports,  recommendations,  and  DOD  actions. 
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DOD  completed  an  extensive  restructuring  of  the  F-35  program  and  we 
reported  in  June  2012  and  again  in  March  2013  that  the  restructuring 
actions  should  lead  to  more  achievable  and  predictable  outcomes,  albeit 
at  higher  cost  and  longer  timeframes  for  testing  and  delivering  capabilities 
to  the  warfighter  than  originally  planned.4 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2010  mandated 
GAO  to  review  the  F-35  acquisition  program  annually  for  6  years.5  This  is 
our  fifth  report  under  that  mandate,  and  in  it,  we  assess  the  program’s  (1 ) 
ongoing  development  and  testing,  (2)  funding  and  long-term  affordability, 
and  (3)  manufacturing  progress. 

To  conduct  our  work,  we  reviewed  and  analyzed  program  briefings, 
management  reports,  program  test  results,  and  internal  DOD  program 
analyses.  We  discussed  key  aspects  of  F-35  performance  with  both 
military  and  private  contractor  test  pilots.  We  interviewed  F-35  program 
and  aircraft  prime  contractor  officials  to  discuss  developmental  testing. 

We  also  collected  developmental  test  plans,  and  data  on  test 
achievements  to  assess  program  progress  through  December  2013.  We 
obtained  current  program  acquisition  and  life-cycle  sustainment  cost 
estimates,  reviewed  the  supporting  documentation  and  discussed  the 
development  of  those  estimates  with  DOD  and  prime  contractor  officials 
instrumental  in  producing  them.  We  toured  F-35  manufacturing  and  test 
facilities  and  obtained  and  analyzed  production  and  supply  chain  data  as 
of  December  2013.  We  assessed  the  reliability  of  DOD  and  contractor 
data  by  reviewing  existing  information  about  the  data,  and  interviewing 
agency  officials  knowledgeable  about  the  data.  We  determined  that  the 
data  were  sufficiently  reliable  for  the  purposes  of  this  report.  We  also 
discussed  ongoing  manufacturing  process  improvements  with  prime 
contractor  and  Defense  Contract  Management  Agency  (DCMA)  officials. 
Appendix  II  contains  a  more  detailed  description  of  our  scope  and 
methodology. 


4GAO,  Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance  Restructuring  and 
Address  Affordability  Risks,  GAO-12-437  (Washington,  D.C.:  June  14,  2012);  and  GAO, 
Joint  Strike  Fighter:  Current  Outlook  Is  Improved,  but  Long-Term  Affordability  Is  a  Major 
Concern,  GAO-13-309  (Washington,  D.C.:  Mar.  11, 2013). 

5Pub.  L.  No.  1 11-84,  §  244  (2009). 
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We  conducted  this  performance  audit  from  June  2013  to  March  2014  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  required  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


As  we  have  reported  in  the  past,  DOD  began  the  F-35  acquisition 
program — also  known  as  the  Joint  Strike  Fighter — in  October  2001, 
without  adequate  knowledge  about  the  aircraft’s  critical  technologies  or  its 
design.6  In  addition,  the  program’s  acquisition  strategy  called  for  high 
levels  of  concurrency  between  development,  testing,  and  production.  As  a 
result,  the  program  encountered  significant  cost  and  schedule  growth  as 
well  as  performance  shortfalls  and  was  restructured  three  times:  first  in 
December  2003,  then  again  in  March  2007,  and  most  recently  in  March 
2012.  The  most  recent  restructuring  was  initiated  in  early  2010,  when  the 
program’s  unit  cost  estimates  exceeded  critical  thresholds  established  by 
statute — a  condition  known  as  a  Nunn-McCurdy  breach.  DOD 
subsequently  certified  to  the  Congress  in  June  2010  that  the  program  was 
essential  to  national  security  and  needed  to  continue.7  DOD  then  began 
efforts  to  significantly  restructure  the  program  and  establish  a  new 
acquisition  program  baseline.  These  restructuring  efforts  continued 
through  2011  and  into  2012,  during  which  the  department  increased  the 
program’s  cost  estimates,  extended  its  testing  and  delivery  schedules, 
and  reduced  near-term  aircraft  procurement  quantities  by  deferring  the 
procurement  of  41 0  aircraft  into  the  future.  The  new  F-35  acquisition 


6GAO-13-309;  GAO-12-437;  GAO,  Joint  Strike  Fighter:  Restructuring  Places  Program  on 
Firmer  Footing,  but  Progress  Still  Lags,  GAO-1 1-325  (Washington,  D.C.:  April  7,  201 1 ). 

7Section  2433  of  title  10  of  the  United  States  Code,  commonly  referred  to  as  Nunn- 
McCurdy,  requires  DOD  to  notify  Congress  whenever  a  major  defense  acquisition 
program’s  unit  cost  experiences  cost  growth  that  exceeds  certain  thresholds.  This  is 
commonly  referred  to  as  a  Nunn-McCurdy  breach.  Significant  breaches  occur  when  the 
program  acquisition  unit  cost  or  procurement  unit  cost  increases  by  at  least  15  percent 
over  the  current  baseline  estimate  or  at  least  30  percent  over  the  original  estimate.  For 
critical  breaches,  when  these  unit  costs  increase  at  least  25  percent  over  the  current 
baseline  estimate  or  at  least  50  percent  over  the  original,  DOD  is  required  to  take 
additional  steps,  including  conducting  an  in-depth  review  of  the  program.  Programs  with 
critical  breaches  must  be  terminated  unless  the  Secretary  of  Defense  certifies  to  certain 
facts  related  to  the  program  and  takes  other  actions,  including  restructuring  the  program. 
10  U.S.C.  §  2433a. 
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program  baseline  was  finalized  in  March  2012,  and  since  that  time  costs 
have  remained  relatively  stable.  Table  1  notes  the  significant  cost, 
quantity,  and  schedule  changes  from  the  original  program  baseline  and 
the  relative  stability  since  the  new  baseline  was  established. 


Table  1:  Changes  in  Reported  F-35  Program  Costand  Quantity  and  Deliveries  2001-2013 


October  2001 
(initial 
baseline) 

March  2012 
(restructured 
baseline) 

March  2013 
(current 
estimates) 

Change 

2001-2012 

Change 

2012-2013 

Expected  quantities  (number  of  aircraft) 

Developmental  quantities 

14 

14 

14 

0% 

0% 

Procurement  quantities 
(U.S.  only) 

2,852 

2,443 

2,443 

-  14 

0 

Total  quantities 

2,866 

2,457 

2,457 

- 14 

0 

Cost  estimates  (then-year  dollars  in  billions) 

Development 

$34.4 

$55.2 

$55.2 

60 

0 

Procurement 

196.6 

335.7 

330.6 

71 

-2 

Military  construction 

2.0 

4.8 

4.6 

140 

-4 

Total  program  acquisition 

233.0 

395.7 

390.4 

70 

-1 

Unit  cost  estimates  (then-year  dollars  in  millions) 

Program  acquisition 

$81 

$161 

$159 

99 

-  1 

Average  procurement 

69 

137 

135 

99 

-  1 

Estimated  delivery  and  production  dates 

Initial  operational  capability 

2010-2012 

TBD 

2015-2018 

NA 

NA 

Full-rate  production 

2012 

2019 

2019 

7  years 

0  years 

Source:  GAO  analysis  of  DOD  data. 

At  the  time  the  new  F-35  acquisition  program  baseline  was  finalized,  it  did 
not  identify  new  initial  operational  capability  (IOC)  dates  for  the  three 
military  services.8  The  following  year  DOD  issued  a  memorandum  noting 
that  Marine  Corps  and  Air  Force  were  planning  to  field  initial  operational 
capabilities  in  2015  and  2016,  respectively,  and  that  the  Navy  planned  to 
field  its  initial  capability  in  2018.  The  memorandum  emphasized  that  the 
Marine  Corps  and  Air  Force  initial  operational  capabilities  would  be 
achieved  with  aircraft  that  possess  initial  combat  capabilities,  and  noted 
that  those  aircraft  would  need  additional  lethality  and  survivability 


8  Initial  operational  capability  is  obtained  when  organizations  or  units  have  received  a 
specified  number  of  systems  and  have  the  ability  to  employ  and  maintain  those  systems. 
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enhancements  to  meet  the  full  spectrum  of  warfighter  requirements  in  the 
future.  These  new  parameters  represented  a  delay  of  5  to  6  years  from 
the  program’s  initial  2001  baseline  and  a  reduction  in  the  capabilities 
expected  at  IOC. 

In  March  2005  we  recommended  that  DOD  implement  an  evolutionary, 
incremental  approach  to  developing  and  fielding  the  F-35 — then  known  as 
the  Joint  Strike  Fighter — to  ensure  that  the  warfighters  would  receive  an 
initial  combat  capability  that,  at  a  minimum,  would  meet  their  most 
immediate  needs.9  Again  in  March  2010,  we  recommended  that  DOD 
identify  the  absolute  minimum  combat  capabilities  that  would  be 
acceptable  by  each  of  the  military  services  to  field  their  initial  operational 
capabilities  and  establish  reasonable,  realistic  timeframes  for  achieving 
those  requirements.10  In  both  instances,  we  noted  that  the  military 
services  should  consider  trading  off  desired  capabilities  in  order  to  more 
rapidly  field  aircraft  with  an  initial  set  of  useable  capabilities  and  that  any 
capabilities  not  needed  to  meet  immediate  warfighting  needs  should  be 
deferred  to  a  future  development  increment.  In  both  instances,  DOD 
agreed  with  the  intent  of  our  recommendation,  but  believed  that  its 
existing  program  management  practices  were  sufficient. 


Testing  of  Critical 
Software  Poses  a 
Significant  Challenge 
to  the  Delivery  of  F- 
35  Warfighting 
Capabilities 


Delays  in  the  testing  of  critical  mission  systems  software  have  put  the 
delivery  of  expected  warfighting  capabilities  to  the  Marine  Corps  at  risk, 
and  could  affect  the  delivery  of  capabilities  to  the  Air  Force  and  Navy  as 
well.  F-35  developmental  flight  testing  is  separated  into  two  key  areas: 
mission  systems  and  flight  sciences.  Mission  systems  testing  is  done  to 
verify  that  the  software  and  systems  that  provide  critical  warfighting 
capabilities  function  properly  and  meet  requirements,  while  flight  science 
testing  is  done  to  verify  the  aircraft’s  basic  flying  capabilities.  In  a  March 
2013  report  we  found  that  development  and  testing  of  mission  systems 
software  was  behind  schedule,  due  largely  to  delayed  software  deliveries, 
limited  capability  in  the  software  when  delivered,  and  the  need  to  fix 
problems  and  retest  multiple  software  versions.  These  same  challenges 
continued  thorough  2013,  and  as  a  result  progress  in  mission  systems 


9  GAO,  Tactical  Aircraft:  Opportunity  to  Reduce  Risk  in  the  Joint  Strike  Fighter  Program 
with  Different  Acquisition  Strategy,  GAO-05-271  (Washington,  D.C.:  March  15,  2005). 

10  GAO,  Joint  Strike  Fighter:  Additional  Costs  and  Delays  Risk  Not  Meeting  Warfighter 
Requirements  on  Time,  GAO-10-382  (Washington,  D.C.:  March  19,  2010). 
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testing  has  been  limited.  The  Director  of  Operational  Test  and  Evaluation 
(DOT&E)  predicts  that  the  delivery  of  expected  warfighting  capabilities  to 
the  Marine  Corps  could  be  delayed  by  as  much  as  13  months.  Delays  of 
this  magnitude  could  also  increase  the  already  significant  concurrency 
between  testing  and  aircraft  procurement  and  result  in  additional  cost 
growth.  Although  mission  systems  testing  is  behind,  the  F-35  program 
has  been  able  to  accomplish  nearly  all  of  its  planned  flight  science 
testing.  The  program  also  continued  to  make  progress  in  addressing  key 
technical  risks,  although  some  of  that  progress  has  been  limited. 


Limited  Progress  in 
Mission  Systems  Software 
Testing  May  Cause  Delays 
and  Add  Cost  to  F-35 
Development 


While  the  F-35  program  was  able  to  accomplish  all  of  the  mission  system 
test  flights  it  had  planned  in  2013,  it  did  not  accomplish  all  of  the  planned 
test  points,11  falling  short  by  11  percent.  The  F-35  program  planned  to  fly 
329  mission  systems  test  flights  and  accomplish  2,817  test  points  in 
2013.  The  program  actually  flew  352  test  flights,  exceeding  the  goal,  but 
only  accomplished  2,518  test  points.  According  to  program  and  contractor 
officials,  slow  progress  in  developing,  delivering,  and  testing  mission 
systems  software  continues  to  be  the  program’s  most  significant  risk 
area.  The  F-35  program  is  developing  and  fielding  mission  systems 
software  capabilities  in  blocks:  (1)  Block  1,  (2)  Block  2A,  (3)  Block  2B,  (4) 
Block  3i,  and  (5)  Block  3F.  Each  subsequent  block  builds  on  the 
capabilities  provided  in  the  preceding  blocks.  Blocks  1  and  2A  provide 
training  capabilities  and  are  essentially  complete,  with  some  final 
development  and  testing  still  underway.  Blocks  2B  and  3i  provide  initial 
warfighting  capabilities  and  are  needed  by  the  Marine  Corps  and  Air 
Force,  respectively,  to  achieve  initial  operational  capability.  Block  3F  is 
expected  to  provide  the  full  suite  of  warfighting  capabilities,  and  is  the 
block  the  Navy  expects  to  have  to  achieve  its  initial  operational  capability. 


Developmental  testing  of  Block  2B  software  is  behind  schedule  and  will 
likely  delay  the  delivery  of  expected  warfighting  capabilities.  The  delivery 
of  this  software  capability  is  of  high  near-term  importance  because  it 
provides  initial  warfighting  capability  for  the  overall  F-35  program,  and  is 
needed  by  the  Marine  Corps  to  field  its  initial  operational  capability  in  July 
2015.  As  of  January  2014,  the  program  planned  to  have  verified  the 
functionality  of  27  percent  of  the  software’s  capability  on-board  the 


11  Flight  test  points  are  specific,  quantifiable  objectives  in  flight  plans  that  are  needed  to 
verify  aircraft  design  and  performance. 
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aircraft,  but  had  only  been  able  to  verify  13  percent.  This  leaves  a 
significant  amount  of  work  to  be  done  before  October  2014,  which  is 
when  the  program  expects  to  complete  developmental  flight  testing  of  this 
software  block.  According  to  DOT&E,  Block  2B  developmental  testing  will 
not  be  completed  as  scheduled  and  could  be  delayed  by  as  much  as  13 
months,  as  the  program  has  had  to  devote  time  and  resources  to  address 
problems  and  completing  development  of  prior  software  blocks.  Delays  of 
this  magnitude  would  mean  that  the  Marine  Corps  will  not  likely  have  all 
of  the  capabilities  it  expects  in  July  2015,  its  planned  initial  operational 
capability  date.  At  this  time  it  is  not  clear  exactly  which  of  the  expected 
capabilities  will  be  available  as  testing  is  still  ongoing.  The  effects  of  these 
delays  compound  as  they  also  put  the  timely  delivery  of  Air  Force  and 
Navy  initial  operational  capabilities  at  risk.  The  Air  Force  expects  to  field 
its  initial  operational  capability  in  August  2016  with  Block  3i  aircraft  that 
possess  the  same  warfighting  capabilities  as  the  Marine  Corps  aircraft, 
but  have  upgraded  computer  hardware.  The  Navy  plans  to  achieve  its 
initial  operational  capability  in  August  2018  with  Block  3F  aircraft  that 
possess  the  full  suite  of  F-35  warfighting  capabilities. 

Program  and  contractor  officials  have  stated  that  while  they  recognize 
that  the  program  faces  software  risks,  they  still  expect  to  deliver  all  of  the 
planned  F-35  software  capabilities  to  the  military  services  as  currently 
scheduled.  However,  given  the  uncertainty  in  mission  systems  software 
testing  and  the  significance  of  the  F-35  to  future  force  structure  plans,  it  is 
important  that  the  military  services  have  a  clear  understanding  of  the 
specific  capabilities  that  they  can  realistically  expect  to  receive,  and  those 
capabilities  that  are  not  likely  to  be  delivered  by  their  initial  operational 
capability  dates — the  first  of  which  is  scheduled  for  July  2015.  In  addition, 
because  the  F-35  is  DOD’s  most  costly  and  ambitious  acquisition 
program,  it  is  important  that  DOD  keep  Congress  informed  about  the 
status  of  the  program.  Without  a  clear  understanding  of  the  specific 
capabilities  that  will  initially  be  delivered,  Congress  and  the  military 
services  may  not  be  able  to  make  fully  informed  resource  allocation 
decisions. 

Delays  in  mission  systems  software  testing  could  also  increase  costs.  As 
currently  planned,  DOD  expects  to  complete  developmental  flight  testing 
in  2017.  If  the  flight  test  schedule  is  extended,  the  program  may  have  to 
retain  testing  and  engineering  personnel  longer  than  currently  expected, 
which  would  increase  development  cost.  DOD  currently  expects  to  have 
invested  $70.2  billion  to  procure  359  aircraft  by  2017  when  developmental 
flight  testing  is  scheduled  to  end.  Our  past  reports  have  concluded  that 
purchasing  aircraft  while  concurrently  conducting  developmental  flight 
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testing  increases  the  risk  that  problems  will  be  discovered  late  in  testing 
and  additional  funding  will  be  needed  to  rework  aircraft  that  have  already 
been  purchased.12  If  F-35  procurement  plans  remain  unchanged  and 
developmental  testing  continues  into  2018,  the  cost  risks  associated  with 
concurrency  will  likely  increase  as  DOD  expects  to  have  invested  $83.4 
billion  in  459  aircraft  by  that  point  in  time. 

The  F-35  contractor  recognizes  that  additional  testing  efficiencies  are 
important  in  order  to  deliver  capabilities  on  schedule  and  cost.  One  way  it 
plans  to  gain  efficiency  is  to  use  test  results  from  one  F-35  variant  to 
close  out  test  points  for  the  other  two  variants  in  instances  in  which  the 
variants  have  common  functions.  According  to  test  officials,  most  mission 
systems  testing  can  be  accomplished  on  any  variant  and  only  a  limited 
amount  of  variant-specific  testing  is  required.  Contractor  officials  pointed 
out  that  this  type  of  efficiency  would  help  mitigate  some  testing  risk,  but 
they  also  recognized  that  it  will  still  be  difficult  to  make  up  the  lost  time  in 
the  test  program.  They  noted  that  delays  in  specific  test  events  generally 
impact  the  entire  test  schedule  because  the  ability  to  conduct  future 
testing  is  often  dependent  on  the  completion  of  the  earlier  events. 


Program  Accomplished 
Most  Flight  Science  Test 
Goals  in  2013 


The  program  accomplished  nearly  all  of  the  flight  sciences  testing, 
including  weapons  testing,  it  had  planned  for  2013.  As  of  December 
2013,  the  program  had  achieved  half  of  the  total  number  of  test  points 
required  to  complete  all  flight  science  testing  for  the  program.  Figure  1 
below  shows  the  number  of  flight  science  test  points  planned  and 
accomplished  for  each  F-35  variant  as  of  December  2013,  and  also 
identifies  the  total  number  of  test  points  required  for  each  variant  to 
complete  flight  science  testing. 


12  GAO-13-309;  GAO-12-437 
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Figure  1:  F-35  Test  Points  Progress  in  2013 
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The  program  made  progress  despite  the  fact  that  flight  testing  was  halted 
twice  at  the  beginning  of  the  year  to  investigate  and  fix  cracks  in  an 
engine  fan  blade  and  leaky  fuel  hoses.  In  addition,  program  and 
contractor  officials  emphasized  that  employee  furloughs  that  occurred  in 
2013,  due  to  mandatory  sequestration,  limited  the  amount  of  flight  testing 
that  could  be  done  during  that  time  as  well.13  Some  of  the  key  flight 
science  and  weapons  testing  accomplishments  included: 

•  Conventional  takeoff  and  landing  variant  -  The  program  successfully 
demonstrated  the  variant’s  ability  to  launch  AIM-120  missiles  from  its 
internal  weapons  bay  and  to  refuel  while  in  flight.  The  program  also 
continued  testing  the  aircraft’s  ability  to  function  at  high  vertical  flight 
angles,  although  program  officials  noted  that  the  testing  took  longer 
than  expected.  As  of  December  2013,  the  program  had  accomplished 


13  The  President  ordered  a  sequestration  of  budgetary  resources  on  March  1 , 201 3  in 
response  to  the  Balanced  Budget  and  Emergency  Deficit  Control  Act  of  1985,  Pub.  L  No. 
99-177,  §251,  as  amended  by  the  Budget  Control  Act  of  2011,  Pub.  L  No.  112-25,  §  302. 
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59  percent  of  its  total  expected  flight  science  test  points  for  this 
variant. 

•  Short  takeoff  and  vertical  landing  variant  -  The  program  successfully 
demonstrated  the  STOVL’s  ability  to  takeoff  vertically,  launch 
weapons  from  its  internal  weapons  bay,  and  dump  fuel  when  needed. 
In  addition,  the  program  conducted  some  testing  of  the  variant  at  sea 
on  an  amphibious  assault  ship — specifically  the  USS  WASP.  As  of 
December  2013,  the  program  had  accomplished  49  percent  of  its  total 
expected  flight  science  test  points  for  this  variant. 

•  Carrier-suitable  variant  -  The  program  began  testing  the  capability  of 
the  aircraft  to  function  at  high  vertical  flight  angles.  In  addition,  the 
program  successfully  demonstrated  the  aircraft’s  ability  to  dump  fuel 
when  needed.  Program  and  contractor  officials  noted  that  the  program 
also  made  progress  to  begin  testing  to  verify  that  the  aircraft’s  new 
arresting  hook  system  could  successfully  catch  a  cable  on  a  set  of 
carrier  arresting  gear  installed  onshore  at  the  Lakehurst  facility.  As  of 
December  2013,  the  program  had  accomplished  43  percent  of  its  total 
expected  flight  science  test  points  for  this  variant. 

Developmental  testing  is  not  the  only  testing  that  the  program  still  has  to 
complete.  The  F-35  program  is  also  scheduled  to  begin  operational 
testing  in  June  2015,  to  determine  that  the  aircraft  variants  can  effectively 
perform  their  intended  missions  in  a  realistic  threat  environment.14 


Program  Progress  in 
Addressing  Key  Technical 
Risks  Has  Been  Mixed 


While  the  F-35  program  made  progress  addressing  some  key  technical 
risks  in  2013,  it  continued  to  encounter  slower  than  expected  progress  in 
developing  the  Autonomic  Logistics  Information  System  (ALIS).  Over 
time,  we  have  reported  on  4  areas  of  technical  and  structural  risk  that  the 
program  identified  during  flight,  ground,  and  lab  testing  that  if  not 
addressed,  could  result  in  substantially  degraded  capabilities  and  mission 
effectiveness.15  In  2013,  we  found  that  the  program  made  the  following 
progress  in  each  of  those  areas: 


•  Helmet  mounted  display  -  provides  flight  data,  targeting,  and  other 
sensor  data  to  the  pilot,  and  is  integral  to  reducing  pilot  workload  and 
achieving  the  F-35’s  concept  of  operations.  The  original  helmet 


14  Interim  Department  of  Defense  Instruction  5000.02  Operation  of  the  Defense 
Acquisition  System,  para.  (5)(d)(11)(b)(2)  (Nov.  25,  2013). 

15  GAO-13-309;  GAO-12-437 
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mounted  display  encountered  significant  technical  deficiencies, 
including  display  jitter,  the  undesired  shaking  of  the  visor  display,  and 
latency,  the  perceivable  lag  that  occurs  in  transmitting  sensor  data, 
and  did  not  meet  warfighter  requirements.  The  program  made 
adjustments  to  the  helmet  design,  including  adding  sensors  to  lessen 
the  display  jitter,  and  redesigning  elements  to  minimize  latency.  The 
program  tested  these  design  changes  in  2013  and  found  that  most  of 
the  technical  deficiencies  had  been  adequately  addressed,  and  that 
the  helmet’s  performance  was  sufficiently  suitable  to  support  Marine 
Corps  initial  operational  capability  in  2015.  DOT&E  and  program  test 
pilots  noted  that  the  current  night  vision  camera  continues  to  have 
problems.  The  program  has  identified  a  new  camera  that  it  believes 
will  address  those  problems,  but  that  camera  has  not  been  fully  tested 
to  verify  its  capabilities. 

•  Arresting  Hook  System  -  allows  the  F-35  carrier-suitable  variant  to 
engage  landing  wires  on  aircraft  carriers,  was  redesigned  after  the 
original  hook  system  was  found  to  be  deficient.  The  program 
determined  that  the  original  hook  assembly  was  not  strong  enough  to 
reliably  catch  the  wire  and  stop  the  airplane.  As  a  result,  the  program 
modified  the  hook  system’s  hydraulic  components,  and  made 
structural  modifications  to  the  plane.  In  March  2013,  the  program 
completed  a  critical  design  review  of  the  hook  system  to  verify  that  the 
new  design  is  sound.  Land  testing  of  the  redesigned  system  has  been 
successful,  and  the  program  anticipates  that  it  will  be  ready  for  carrier 
testing  in  October  2014. 

•  Durability  -  structural  and  durability  testing  of  the  aircraft  continued  in 
2013,  and  the  program  completed  the  first  round  of  this  testing  on  all 
three  variants.  The  conventional  takeoff  and  landing  variant  and  the 
short  takeoff  and  vertical  landing  variants  have  also  started  their 
second  round  of  testing.  During  this  second  round  of  testing,  the  short 
takeoff  and  vertical  landing  test  aircraft  developed  bulkhead  cracks  at 
the  equivalent  of  17  years  of  service  life.  Contractor  officials  noted 
that  they  were  working  to  develop  a  solution  to  those  cracks,  but  the 
total  cost  and  schedule  impacts  of  these  bulkhead  cracks  are 
unknown  at  this  time. 

•  Autonomic  Logistics  Information  System  -  an  important  tool  to 
predict  and  diagnose  maintenance  and  supply  issues,  automate 
logistics  support  processes  and  provide  decision  aids  aimed  at 
reducing  life-cycle  sustainment  costs  and  improving  force  readiness. 
ALIS  is  being  developed  and  fielded  in  increments.  In  2013,  the 
program  had  to  release  an  update  to  the  first  increment  because 
problems  were  discovered  after  the  increment  was  released  to  the 
testing  locations.  The  additional  time  to  develop  and  field  this  update 
will  likely  delay  the  delivery  of  future  increments.  The  program 
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completed  site  activation  of  ALIS  systems  at  some  training  and  testing 
locations,  and  is  in  the  process  of  adding  capabilities  and  maturing 
ALIS  in  a  second  increment  to  support  the  Marine  Corps’  initial 
operational  capability.  DOT&E  notes  that,  although  the  second 
increment  is  scheduled  to  be  delivered  in  time  to  support  the  Marine 
Corps’  initial  operational  capability,  there  is  no  margin  for  error  in  the 
development  schedule.  Testing  of  this  ALIS  increment  is  about  two 
months  behind  largely  due  to  a  lack  of  test  facilities.  Program  officials 
note  that  they  are  in  the  process  of  adding  facilities.  The  third,  and 
final,  increment  of  ALIS  that  provides  full  capability  is  not  expected  to 
be  released  until  2016. 


Program  Funding 
Projections  and  Unit 
Cost  Targets  May  Not 
Be  Achievable 


The  F-35  program’s  high  projected  annual  acquisition  funding  levels 
continue  to  put  the  program’s  long-term  affordability  at  risk.  Currently  the 
acquisition  program  requires  $12.6  billion  per  year  through  2037,  which 
does  not  appear  to  be  achievable  given  the  current  fiscal  environment. 
The  program  is  reducing  unit  costs  to  meet  targets,  but  a  significant 
amount  of  additional  cost  reduction  is  needed  if  it  expects  to  meet  those 
targets  before  the  beginning  of  full  rate  production — currently  scheduled 
for  2019.  Additionally,  the  most  recent  life-cycle  sustainment  cost 
estimate  for  the  F-35  fleet  is  more  than  $1  trillion,  which  DOD  officials 
have  deemed  unaffordable.  The  program’s  long  term  sustainment 
estimates  reflect  assumptions  about  key  cost  drivers  that  the  program 
does  not  control,  including  fuel  costs,  labor  costs,  and  inflation  rates.  The 
program  is  also  focusing  on  product  reliability,  which  is  something  that  the 
program  can  control,  and  something  we  have  found  in  our  prior  best 
practices  work  to  be  a  key  to  driving  down  sustainment  costs.  According 
to  program  reliability  data,  each  F-35  variant  was  tracking  closely  to  its 
reliability  plan  as  of  December  2013,  although  the  program  has  a  long 
way  to  go  to  achieve  its  reliability  goals. 


Long-term  Acquisition 
Funding  Projections  Pose 
Affordability  Challenges 


The  overall  affordability  of  the  F-35  acquisition  program  remains  a 
significant  concern.  As  of  March  2013,  the  program  office  estimated  that 
the  total  acquisition  cost  will  be  $390.4  billion.  DOD’s  estimated  annual 
funding  levels  to  finish  development  and  procurement  of  the  F-35  are 
shown  in  figure  2. 
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Figure  2:  Budgeted  Development  and  Procurement  Costs  by  Service,  2014-2037  (dollars  in  then  year  billions) 
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Source:  GAO  analysis  of  DOD  data. 


From  fiscal  years  2014  to  2018,  DOD  plans  to  increase  development  and 
procurement  funding  for  the  F-35  from  around  $8  billion  to  around  $13 
billion,  an  investment  of  more  than  $50  billion  over  that  5-year  period. 

This  build-up  will  occur  during  years  of  potential  reductions  in  DOD’s 
budget  as  a  result  of  sequestration.  From  fiscal  year  2014  through  fiscal 
year  2037,  the  program  projects  that  it  will  require,  on  average, 
development  and  procurement  funding  of  $12.6  billion  per  year,  with 
several  peak  years  at  around  $15  billion.  Such  a  high  average  annual 
cost  requirement  poses  affordability  risks.  At  $12.6  billion  a  year,  the  F-35 
acquisition  program  alone  would  consume  around  one-quarter  of  all  of 
DOD’s  annual  major  defense  acquisition  funding.  Therefore,  any  change 
in  F-35  funding  is  likely  to  affect  DOD’s  ability  to  fully  fund  its  other  major 
acquisition  programs.  In  addition,  maintaining  this  level  of  sustained 
funding  will  be  difficult  in  a  period  of  declining  or  flat  defense  budgets  and 
competition  with  other  large  acquisition  programs  such  as  the  KC-46 
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tanker  and  a  new  bomber.  These  costs  do  not  include  the  costs  to 
operate  and  maintain  the  F-35s  as  they  are  produced  and  fielded. 


Recognizing  the  affordability  challenges  posed  by  the  F-35  program,  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
established  affordability  unit  cost  targets  for  each  F-35  variant  to  be  met 
by  the  start  of  full  rate  production  in  2019.  The  program  is  likely  to  be 
challenged  to  meet  those  targets,  as  the  three  variants  still  require 
anywhere  from  $41  million  to  $49  million  in  unit  cost  reductions  (see  table 
2).  In  addition,  the  program’s  current  funding  and  quantity  projections 
indicate  that  unit  costs  in  2019  could  actually  be  higher  than  the  targets. 
The  Under  Secretary  issued  a  memorandum  in  April  2013  explaining  that 
affordability  constraints  are  intended  to  force  prioritization  of 
requirements,  drive  performance  and  cost  trades,  and  ensure  that 
unaffordable  programs  do  not  enter  the  acquisition  process.  The 
memorandum  goes  on  to  state  that  “if  affordability  caps  are  breached, 
costs  must  be  reduced  or  else  program  cancelation  can  be  expected.” 


Table  2:  Comparison  of  2013  Unit  Costs  and  2019  Unit  Cost  Goals  (dollars  in  then 
year  millions) 

F-35  variant 

Unit  cost  of 
aircraft  procured 
in  2013 

Unit  cost  target  of 
aircraft  to  be 
procured  in  2019a 

Amount 
currently  over 
cost  targets 

Conventional  takeoff  and 
vertical  landing 

$124.8 

$83.4 

$41.4 

Short  takeoff  and  vertical 
landing 

$156.8 

$108.1 

$48.7 

Carrier-suitable  variant 

$142.6 

$93.3 

$49.3 

Source:  GAO  analysis  of  DOD  data. 


These  targets  are  based  upon  planning  assumptions  reflected  in  the  FY  2013  President’s  Budget 
and  the  201 1  projection  for  international  partner  procurement.  If  there  are  subsequent  changes  to 
either  the  U.S.  or  international  partner  procurement  quantities  the  program  will  isolate  the  effect  that 
this  has  on  the  targets  as  a  factor  that  is  not  within  their  control. 


The  F-35  program  made  progress  this  year  in  decreasing  the  unit  costs  of 
the  conventional  take-off  and  landing  and  carrier-suitable  variants,  but  the 
unit  cost  of  the  short  takeoff  and  vertical  landing  variant  increased  by 
nearly  $10  million.  According  to  program  officials,  the  unit  cost  of  the 
short  takeoff  and  vertical  landing  variant  increased  because  the  program 
had  to  delay  the  procurement  of  a  number  of  aircraft  into  the  future,  which 
reduced  near-term  quantities  and  made  each  individual  unit  more  costly, 
and  engine  costs  were  higher  than  originally  estimated.  There  is  still 
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uncertainty  surrounding  these  estimates  depending  upon  how  DOD 
chooses  to  implement  sequestration  in  future  budgets. 


Estimated  Sustainment 
Costs  Remain  High  but 
Emphasis  on  Improving 
Reliability  Could  Reduce 
Costs 


In  addition  to  the  concerns  about  the  affordability  of  the  F-35  acquisition 
program,  there  are  also  significant  concerns  about  the  cost  of  operating 
and  supporting  the  F-35  fleet  over  the  coming  decades.  Currently,  the 
Cost  Assessment  and  Program  Evaluation  (CAPE)  office,  within  the 
Office  of  the  Secretary  of  Defense,  estimates  that  the  cost  to  operate  and 
support  the  fleet  over  30  years  is  likely  to  exceed  $1  trillion,  which  is  3 
times  higher  than  what  was  projected  when  the  development  program 
began  in  2001.  CAPE’s  estimates  also  indicate  that  F-35  operations  and 
support  costs  could  surpass  the  average  cost  of  legacy  aircraft  by  40 
percent  or  more,  when  original  estimates  indicated  that  the  F-35  would 
cost  less  than  the  legacy  aircraft.  Program  officials  recently  stated  that 
their  estimates  indicate  that  operation  and  support  costs  are  likely  to  be 
closer  to  $860  billion,  and  not  the  $1  trillion  estimated  by  CAPE. 
According  to  CAPE,  program,  and  contractor  officials,  F-35  sustainment 
cost  estimates  differ  as  the  assumed  future  values  for  key  cost  drivers, 
like  inflation  rates  and  fuel  costs,  vary  among  cost  estimators.  CAPE 
officials  emphasize  that  the  difference  between  cost  estimates  is  almost 
entirely  attributable  to  the  use  of  different  inflation  indices.  Table  3  below 
lists  the  top  cost  drivers  in  the  F-35  operation  and  sustainment  estimates. 
While  it  is  important  for  the  program  to  consider  potential  reductions  or 
increases  in  these  variables  listed  below  as  it  estimates  the  F-35’s  long¬ 
term  operation  and  sustainment  costs,  some  of  those  variables  can  be 
directly  controlled  by  the  program  office  while  others  like  inflation  rates 
and  fuel  costs  cannot. 
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Table  3:  Cost  Drivers  That  Affect  F-35  Operation  and  Sustainment  Cost  Estimates 


Input 

Cost  driver 

Risk/Uncertainty 

On-site  repairs  and  number  of 
spares 

Cost  of  providing  maintenance  with 
spares  on  site 

Services  are  in  the  process  of  determining  the 
location  of  F-35  repair  work  as  well  as  the  number  of 
spare  parts  needed.  The  services  decide  locations, 
but  the  number  of  spares  may  be  determined  based 
on  other  factors  outside  of  military  control. 

Fuel  costs 

Cost  of  fuel  to  fly  planes  at  expected 
hours 

Fuel  usage  is  currently  based  on  data  gathered  from 
developmental  test  aircraft.  Jet  fuel  costs  are 
volatile.  The  cost  of  jet  fuel  grew  769  percent  for 
American  private  flight  carriers  between  1977  and 
2013.  Although  the  services  cannot  control  the  cost 
directly,  they  can  determine  the  number  of  hours 
flown  which  affects  the  amount  of  fuel  used. 

Mission  personnel 

Cost  of  employing  and  training  personnel 

The  services  have  not  yet  defined  the  number  of 
personnel  needed. 

Government  versus  contractor 
support  labor  rates 

Percentage  of  work  conducted  by 
government  versus  contractor  personnel; 
cost  of  reimbursement 

The  F-35  program  office  has  not  yet  made  a 
decision  on  the  percentage  of  work  to  be  conducted 
by  government  versus  contractor  personnel. 

Number  and  size  of  squadrons 

Includes  the  number/size  of  squadrons, 
and  squadron  personnel  estimates 

The  military  services  are  currently  considering  the 
number  and  size  of  squadrons. 

Equipment  costs  and  reliability 

Costs  of  equipment  needed  to  keep 
fleets  operational 

Development  is  ongoing,  so  actual  aircraft  reliability 
is  not  yet  known.  The  services  have  little  control 
over  equipment  costs,  but  can  impact  reliability  as 
the  system  is  developed. 

Number  of  planes  and  flight  hours 

Air  Force:  1 ,763  planes  at  250  hours; 
Navy:  340  planes  at  316  hours;  Marine 
Corps:  340  planes  at  302  hours 

The  services  have  decreased  their  anticipated  flight 
hours  throughout  development,  and  may  do  so 
again  in  the  future. 

Inflation  rates 

Annual  Inflation  indices  used  by  the 
Department  of  Defense 

These  rates  are  adjusted  by  the  Office  of 

Management  and  Budget  annually. 

Source:  GAO  Analysis  of  DOD,  contractor,  and  aircraft  industry  data. 


The  F-35  program  office  and  prime  contractor  are  working  to  make  the 
long-term  program  more  affordable.  Starting  in  September  2013,  they 
established  a  sustainment  cost  initiative  team  to  meet  regularly  and 
discuss  options  for  driving  down  sustainment  costs.  According  to 
contracting  officials,  they  also  developed  a  management  team  dedicated 
to  improving  the  aircraft’s  prognostics  and  health  management  system. 
Additionally,  the  program  is  awaiting  the  results  of  a  business  case 
analysis  of  the  costs  and  benefits  of  various  sustainment  options.  The  first 
phase  of  that  analysis,  completed  in  2012,  found  that  relying  on 
government  personnel  for  sustainment  processes  would  be  less  costly 
than  having  contractors  do  the  work.  The  second  phase,  which  began  in 
2013,  is  examining  multiple  aspects  of  the  F-35  sustainment  strategy, 
including  the  program’s  approach  to  maintenance  management  and 
supply  chain  support.  The  findings  of  this  second  phase  are  expected  to 
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identify  affordability  opportunities  and  areas  for  additional  future  analysis. 
Those  findings  are  expected  to  be  provided  to  the  program  by  March 
2014. 

As  the  program  faces  key  decisions  about  its  F-35  operation  and  support 
strategy  reliability  is  still  a  significant  concern.  Our  past  work  has  found 
that  weapon  system  operating  and  support  costs  are  directly  correlated  to 
weapon  system  reliability,  which  is  something  the  program  can  affect.16 
We  found  that  lower  reliability  causes  an  imbalance  in  the  relationship 
between  readiness  and  operating  costs,  and  lends  toward  the  need  for 
high  costs  to  maintain  readiness,  as  seen  in  figure  3  below.  We  also 
previously  found  that  reliability  problems  identified  in  DOD  weapon 
systems  resulted  in  cost  overruns  and  schedule  delays.17 


Figure  3:  Relationship  among  Readiness,  Reliability,  and  Sustainment  Costs 

«— ►  Readiness  requirement  ► 

Best  practice 
outcome: 

Achievement  of 
high  reliability 
rates  during 
development 
allows  lower 
operating  and 
support  costs 
once  fielded 

Source:  GAO. 


Poor  outcome: 

Achievement  of 
low  reliability 
rates  during 
development 
requires  higher 
operating  and 
support  costs 
once  fielded. 


Operating  and 
support  costs 


DOD  and  the  contractor  use  various  measures  to  track  and  improve  F-35 
reliability,  including  average  flying  hours  between  failures,  which  is 
defined  as  the  number  of  flying  hours  achieved  divided  by  the  number  of 
failures  incurred.  As  indicated  in  figure  4,  the  conventional  takeoff  and 
landing  variant  and  the  short  takeoff  and  vertical  landing  variant  were  not 
meeting  expected  reliability  as  of  September  2013,  while  the  carrier- 
suitable  variant  was  performing  better  than  expected. 


16  See  GAO,  Best  Practices:  Setting  Requirements  Differently  Could  Reduce  Weapon 
Systems’  Total  Ownership  Costs,  GAO-03-57  (Washington,  D.C.:  Feb.  11, 2003). 

17  See  GAO,  Best  Practices:  Increased  Focus  on  Requirements  and  Oversight  Needed  to 
Improve  DOD’s  Acquisition  Environment  Weapon  System  Quality,  GAO-08-294 
(Washington,  D.C.:  Feb.  1, 2008). 
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Figure  4:  F-35  Average  Flying  Hours  between  Failure  Rates  by  Variant 


Number  of  hours 


Conventional  take-off 
and  landing 


Short  take-off  vertical 
landing 


Carrier  variant 


Demonstrated  as  of  September  2013 
Planned  by  September  201 3 
Goal  at  maturity 


Source:  GAO  analysis  of  DOD  data. 


DOT&E’s  recent  report  noted  concerns  about  the  program’s  ability  to 
achieve  its  reliability  goals  by  the  time  each  of  the  F-35  variants  reaches 
maturity — defined  as  75,000  flight  hours  for  the  CTOL  and  STOVL 
variants;  and  50,000  flight  hours  for  the  CV.  DOT&E  also  noted  that  the 
F-35  design  is  becoming  more  stable,  and  although  the  program  still  has 
a  large  number  of  flight  hours  to  go  until  system  maturity,  additional 
reliability  growth  is  not  likely  to  occur  without  a  focused,  aggressive  and 
well-resourced  effort. 


F-35  Aircraft 
Manufacturing 
Efficiency  Continues 
to  Improve 


F-35  manufacturing  has  improved  and  the  contractor’s  management  of  its 
suppliers  is  evolving.  As  the  number  of  aircraft  in  production  has 
increased,  learning  has  taken  place  and  manufacturing  efficiency  has 
improved.  For  example,  the  prime  contractor  has  seen  reductions  in 
overall  labor  hours  needed  to  manufacture  the  aircraft.  The  number  of  F- 
35  aircraft  produced  and  delivered  annually  by  the  prime  contractor  has 
steadily  increased  since  the  first  low  rate  production  aircraft  were 
delivered  in  201 1 .  In  2013,  the  contractor  delivered  35  aircraft  to  the 
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government,  5  more  than  it  delivered  in  2012  and  26  more  than  it 
delivered  in  201 1 .  The  prime  contractor  has  put  in  place  a  supplier 
management  system  to  oversee  key  supplier  performance  allowing  them 
to  identify  poor  performers  and  take  appropriate  action  to  address  issues 
such  as  part  shortages  and  poor  quality.  According  to  contractor  officials, 
actions  taken  as  a  result  of  this  system  contributed  to  improvements  in 
supplier  performance  over  the  past  year. 


Program  Manufacturing 
Processes  Continue  to 
Show  Improvement 


The  prime  contractor  continues  to  gain  efficiencies  in  the  manufacturing 
process  as  it  learns  more  about  manufacturing  the  aircraft.  Reductions  in 
the  amount  of  time  spent  on  work  completed  out  of  its  specific  work 
station  have  contributed  to  overall  labor  hour  reductions.  Aircraft  delivered 
in  2012  averaged  about  93  hours  of  out  of  station  work  per  aircraft,  while 
in  2013  about  8  hours  of  out  of  station  work  were  expended  per  aircraft 
on  average.  While  these  gains  in  efficiency  have  moved  the  program 
closer  to  meeting  its  established  labor  hour  goals,  there  is  still  a  long  way 
to  go.  In  2013,  the  prime  contractor  was  unable  to  reach  labor  hour  goals 
for  both  the  CTOL  and  STOVL  variants.18  By  the  end  of  2014,  the  prime 
contractor  expects  to  significantly  reduce  the  average  labor  hours  to 
produce  the  CTOL  and  STOVL  variants.  However,  in  order  to  meet  its 
goal,  the  program  will  have  to  reduce  the  average  number  of  hours  per 
aircraft  for  CTOL  production  by  nearly  20,000  and  the  average  number  of 
hours  per  aircraft  for  STOVL  production  by  more  than  14,000  over  the 
next  year.  While  challenging,  this  goal  appears  to  be  achievable  given 
that  the  program  has  been  able  to  reduce  the  average  labor  hours  for 
CTOL  and  STOVL  production  by  more  than  20,000  hours  annually  since 
201 1 .  Figure  5  identifies  the  prime  contractor’s  trend  in  reduction  of  labor 
hours  since  the  beginning  of  low-rate  initial  production  as  well  as  the 
contractor’s  plans  for  2014. 


18  The  CV  recently  entered  production,  limiting  the  data  that  were  available  for  trend 
analysis.  As  a  result  it  has  been  excluded  in  this  assessment. 
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Figure  5:  Average  Labor  Hours  Per  Aircraft  Since  Beginning  Low-Rate  Production 
and  2014  Goal 


Labor  hours  (in  thousands) 

140 


2011  2012  2013  2014  Goal 


Conventional  take-off  and  landing 
Short  take-off  and  landing 


Source:  GAO  analysis  of  DOD  data. 


As  manufacturing  efficiency  has  improved,  the  prime  contractor  has  also 
been  able  to  increase  throughput,  delivering  more  aircraft  year  over 
year — 9  in  2011,  30  in  2012,  and  35  in  2013.  Over  the  past  year,  the 
prime  contractor  continued  to  deliver  aircraft  closer  to  contracted  delivery 
dates.  Last  year  we  found  that  deliveries  averaged  1 1  months  late,  but 
had  improved  considerably.19  We  found  similar  results  this  year  with 
deliveries  in  2013  averaging  about  5  months  late.  While  deliveries  in  2013 
were  later  than  planned,  the  trend  continued  to  move  in  the  right  direction. 
Figure  6  tracks  the  actual  delivery  dates  against  the  dates  specified  in  the 
contract. 


19  GAO-13-309. 
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Figure  6:  Trend  of  F-35  Actual  Delivery  Dates  Compared  to  Contractual  Delivery  Dates 
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Source:  GAO  analysis  of  DOD  data. 


The  improvements  in  throughput  in  2013  were  achieved  despite  setbacks 
that  kept  the  contractor  from  delivering  aircraft  for  several  months  during 
the  year.  Those  setbacks  included  a  fleet  grounding  due  to  engine 
problems  discovered  in  testing  and  the  need  to  repave  the  runway  used 
by  the  prime  contractor.  The  prime  contractor  took  actions  that  helped 
mitigate  the  impacts  of  these  events  and  as  a  result  delivered  35  aircraft 
in  2013 — one  aircraft  short  of  its  plan. 


Prime  Contractor  Supplier  The  prime  contractor  is  responsible  for  managing  a  complex  supply  chain 
Management  System  made  up  of  a  large  number  of  national  and  international  suppliers. 

Currently,  the  prime  contractor  oversees  about  1 ,500  domestic  suppliers 
and  80  international  suppliers  spread  across  1 1  countries.  Figure  7 
identifies  those  countries  participating  in  F-35  production.  The  number  of 
suppliers  has  grown  significantly  over  the  past  three  years.  Since  201 1 , 
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153  new  suppliers  have  been  added  to  the  supply  chain,  67  of  those  were 
added  between  2012  and  2013.  The  prime  contractor  expects  more 
suppliers  to  be  added  as  production  increases  and  the  program 
progresses. 


Figure  7:  Countries  Currently  Participating  in  F-35  Production 
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Source:  GAO  presentation  of  DOD  data;  Map  Resources  (map). 


The  prime  contractor’s  insight  into  the  performance  of  52  of  its  key 
suppliers  through  its  Supplier  Integration  Management  system  has  led  to 
actions  that  have  improved  supplier  performance.  The  system  tracks 
supplier  performance  data  in  23  areas  including,  but  not  limited  to,  cost 
growth,  parts  shortage  occurrences,  and  the  number  of  corrective  action 
reports  filed.  That  data  is  reviewed  and  scored  on  a  monthly  basis,  with 
each  supplier  receiving  an  overall  score  based  on  their  performance. 
According  to  the  prime  contractor,  a  score  of  80  or  above,  out  of  100,  is 
considered  good  performance.  All  52  of  the  key  suppliers  tracked  using 
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this  system  were  considered  good  performers  as  of  December  2013.  In 
addition,  15  showed  improvement  in  performance  over  the  last  year. 
According  to  contractor  officials,  the  system  identifies  poor  performers 
who  are  counseled  and  corrective  actions  are  identified  and  implemented. 
For  example,  according  to  officials  from  the  prime  contractor,  one  supplier 
was  identified  through  this  process  as  having  a  large  number  of  parts  that 
did  not  conform  to  specifications.  The  prime  contractor  held  meetings  with 
and  provided  direction  to  that  supplier.  As  a  result  of  the  prime 
contractor’s  actions,  the  supplier’s  performance  has  improved  over  the 
last  year.  In  addition,  officials  from  the  prime  contractor  have  identified 
part  shortages — parts  that  are  late  to  production  need — as  a  major 
concern.  The  number  of  part  shortages  has  slightly  increased  over  the 
last  year.  Although  the  root  cause  of  the  shortages  is  still  being  assessed, 
officials  from  the  prime  contractor  stated  they  are  currently  working  on 
ways  to  improve  part  availability. 


Conclusions 


Since  the  F-35  program  restructuring  was  completed  in  March  2012, 
acquisition  cost  and  schedule  estimates  have  remained  relatively  stable, 
and  the  program  has  made  progress  in  key  areas.  However,  persistent 
software  problems  have  slowed  progress  in  mission  systems  flight 
testing,  which  is  critical  to  delivering  the  warfighting  capabilities  expected 
by  the  military  services.  These  persistent  delays  put  the  program’s 
development  cost  and  schedule  at  risk.  As  a  result,  DOT&E  now  projects 
that  the  warfighting  capabilities  expected  by  the  Marine  Corps  in  July 
2015,  will  not  likely  be  delivered  on  time,  and  could  be  delayed  as  much 
as  13  months.  This  means  that  the  Marine  Corps  may  initially  receive  less 
capable  aircraft  than  it  expects,  and  if  progress  in  mission  systems 
software  testing  continues  to  be  slower  than  planned,  Air  Force  and  Navy 
initial  operational  capabilities  may  also  be  affected.  The  program  may 
also  have  to  extend  its  overall  developmental  flight  test  schedule,  which 
would  increase  concurrency  between  testing  and  production  and  could 
result  in  additional  development  cost  growth.  In  addition  to  software 
concerns,  the  current  funding  plans  may  be  unaffordable,  given  current 
budget  constraints.  This  situation  could  worsen  if  unit  cost  targets  are  not 
met.  Finally,  the  estimated  cost  of  operating  and  supporting  the  fleet  over 
its  life-cycle  continues  to  be  high  and  could  increase  further  if  aircraft 
reliability  goals  are  not  met. 

DOD  has  already  made  a  number  of  difficult  decisions  to  put  the  F-35  on 
a  more  sound  footing.  More  such  decisions  may  lie  ahead.  For  example, 
if  software  testing  continues  to  be  delayed,  if  funding  falls  short  of 
expectations,  or  if  unit  cost  targets  cannot  be  met,  DOD  may  have  to 
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make  decisions  about  whether  to  proceed  with  production  as  planned 
with  less  capable  aircraft  or  to  alter  the  production  rate.  Also,  if  reliability 
falls  short  of  goals,  DOD  may  have  to  make  decisions  about  other  ways 
to  reduce  sustainment  costs,  such  as  reduced  flying  hours.  Eventually, 
DOD  will  have  to  make  contingency  plans  for  these  and  other  issues.  At 
this  point,  we  believe  the  most  pressing  issue  is  the  effect  software 
testing  delays  are  likely  to  have  on  the  capabilities  of  the  initial 
operational  aircraft  that  each  military  service  will  receive.  In  order  to  make 
informed  decisions  about  weapon  system  investments  and  future  force 
structure,  it  is  important  that  Congress  and  the  services  have  a  clear 
understanding  of  the  capabilities  that  the  initial  operational  F-35  aircraft 
will  possess. 

Recommendation  for 
Executive  Action 

Due  to  the  uncertainty  surrounding  the  delivery  of  F-35  software 
capabilities,  we  recommend  that  the  Secretary  of  Defense  conduct  an 
assessment  of  the  specific  capabilities  that  realistically  can  be  delivered 
and  those  that  will  not  likely  be  delivered  to  each  of  the  services  by  their 
established  initial  operational  capability  dates.  The  results  of  this 
assessment  should  be  shared  with  Congress  and  the  military  services  as 
soon  as  possible  but  no  later  than  July  2015. 

Agency  Comments 

DOD  provided  comments  on  a  draft  of  this  report,  which  are  reprinted  in 
appendix  III.  DOD  concurred  with  our  recommendation. 

We  are  sending  copies  of  this  report  to  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Secretaries  of  the  Air  Force, 
Army,  and  Navy;  the  Commandant  of  the  Marine  Corps;  and  the  Director 
of  the  Office  of  Management  and  Budget.  The  report  is  available  at  no 
charge  on  the  GAO  website  at  http://www.gao.gov. 
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If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-4841  orsullivanm@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  Staff  members  making  key  contributions  to 
this  report  are  listed  in  appendix  IV. 


Michael  J.  Sullivan 
Director 

Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  Carl  Levin 
Chairman 

The  Honorable  James  Inhofe 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Richard  Durbin 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  P.  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  1: 

Prior  GAO  Reports  and  DOD 

Responses 

GAO  report 

Est.  dev.  Costs  dev. 

Length 

Aircraft  unit  cost  Key  program  event 

Primary  GAO  message 

DOD  response  and  actions 

2001 

GAO-02-39 

$34.4  Billion 

1 0  years 
$69  Million 

Start  of  system 
development  and 
demonstration  approved. 

Critical  technologies  needed 
for  key  aircraft  performance 
elements  are  not  mature. 
Program  should  delay  start  of 
system  development  until 
critical  technologies  are 
mature  to  acceptable  levels. 

DOD  did  not  delay  start  of  system 
development  and  demonstration 
stating  technologies  were  at 
acceptable  maturity  levels  and 
will  manage  risks  in  development. 

2005 

GAO-05-271 

$44.8  Billion 

1 2  years 
$82  Million 

The  program  undergoes 
re-plan  to  address  higher 
than  expected  design 
weight,  which  added  $7 
billion  and  18  months  to 
development  schedule. 

We  recommend  that  the 
program  reduce  risks  and 
establish  executable 
business  case  that  is 
knowledge-based  with  an 
evolutionary  acquisition 
strategy. 

DOD  partially  concurred  but  does 
not  adjust  strategy,  believing  that 
their  approach  is  balanced 
between  cost,  schedule  and 
technical  risk. 

2006 

GAO-06-356 

$45.7  Billion 

1 2  years 
$86  Million 

Program  sets  in  motion 
plan  to  enter  production 
in  2007  shortly  after  first 
flight  of  the  non¬ 
production  representative 
aircraft. 

The  program  plans  to  enter 
production  with  less  than  1 
percent  of  testing  complete. 
We  recommend  program 
delay  investing  in  production 
until  flight  testing  shows  that 
JSF  performs  as  expected. 

DOD  partially  concurred  but  did 
not  delay  start  of  production 
because  they  believe  the  risk 
level  was  appropriate. 

2007 

GAO-07-360 

$44.5  Billion 

1 2  years 
$104  Million 

Congress  reduced 
funding  for  first  two  low- 
rate  production  buys 
thereby  slowing  the  ramp 
up  of  production. 

Progress  is  being  made  but 
concerns  remained  about 
undue  overlap  in  testing  and 
production.  We  recommend 
limits  to  annual  production 
quantities  to  24  a  year  until 
flying  quantities  are 
demonstrated. 

DOD  non-concurred  and  felt  that 
the  program  had  an  acceptable 
level  of  concurrency  and  an 
appropriate  acquisition  strategy. 

2008 

GAO-08-388 

$44.2  Billion 

1 2  years 
$104  Million 

DOD  implemented  a  Mid- 
Course  Risk  Reduction 
Plan  to  replenish 
management  reserves 
from  about  $400  million 
to  about  $1  billion  by 
reducing  test  resources. 

We  believe  new  plan 
increased  risks  and  DOD 
should  revise  it  to  address 
concerns  about  testing, 
management  reserves,  and 
manufacturing  concerns.  We 
determined  that  the  cost 
estimate  was  not  reliable  and 
that  a  new  cost  estimate  and 
schedule  risk  assessment  is 
needed. 

DOD  did  not  revise  risk  plan  or 
restore  testing  resources,  stating 
that  they  will  monitor  the  new 
plan  and  adjust  it  if  necessary. 
Consistent  with  a  report 
recommendation,  a  new  cost 
estimate  was  prepared,  but  DOD 
did  not  conduct  a  risk  and 
uncertainty  analysis. 
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Appendix  I:  Prior  GAO  Reports  and  DOD 
Responses 


Est.  dev.  Costs  dev. 

Length 

GAO  report  Aircraft  unit  cost  Key  program  event  Primary  GAO  message  DOD  response  and  actions 


2009  $44.4  Billion 

GAO-09-303  1 3  years 

$104  Million 


The  program  increased 
the  cost  estimate  and 
adds  a  year  to 
development  but 
accelerated  the 
production  ramp  up. 
Independent  DOD  cost 
estimate  (JET  I)  projects 
even  higher  costs  and 
further  delays. 


Moving  forward  with  an 
accelerated  procurement 
plan  and  use  of  cost 
reimbursement  contracts  is 
very  risky.  We  recommended 
the  program  report  on  the 
risks  and  mitigation  strategy 
for  this  approach. 


DOD  agreed  to  report  its 
contracting  strategy  and  plans  to 
Congress  and  conduct  a 
schedule  risk  analysis.  The 
program  reported  completing  the 
first  schedule  risk  assessment 
with  plans  to  update  semi¬ 
annually.  The  Department 
announced  a  major  program 
reducing  procurement  and 
moving  to  fixed-price  contracts. 


2010  $49.3  Billion 

GAO- 1 0-382  1 5  years 

$112  Million 


2011  $51.8  Billion 

GAO- 11-325  16  years 

$133  Million 


The  program  was 
restructured  to  reflect 
findings  of  recent 
independent  cost  team 
(JET  II)  and  independent 
manufacturing  review 
team.  As  a  result, 
development  funds 
increased,  test  aircraft 
were  added,  the 
schedule  was  extended, 
and  the  early  production 
rate  decreased. 


Restructuring  continued 
with  additional 
development  cost 
increases;  schedule 
growth;  further  reduction 
in  near-term  procurement 
quantities;  and 
decreased  the  rate  for 
future  production.  The 
Secretary  of  Defense 
placed  the  STOVL 
variant  on  a  two-year 
probation;  decoupled 
STOVL  from  the  other 
variants;  and  reduced 
STOVL  production  plans 
for  fiscal  years  201 1  to 
2013. 


Costs  and  schedule  delays 
inhibit  the  program’s  ability  to 
meet  needs  on  time.  We 
recommend  the  program 
complete  a  full 

comprehensive  cost  estimate 
and  assess  warfighter  and 
IOC  requirements.  We 
suggest  that  Congress 
require  DOD  to  tie  annual 
procurement  requests  to 
demonstrated  progress. 


The  restructuring  actions  are 
positive  and  if  implemented 
properly,  should  lead  to  more 
achievable  and  predictable 
outcomes.  Concurrency  of 
development,  test,  and 
production  is  substantial  and 
provides  risk  to  the  program. 
We  recommended  the 
program  maintain  funding 
levels  as  budgeted;  establish 
criteria  for  STOVL  probation; 
and  conduct  an  independent 
review  of  software 
development,  integration,  and 
test  processes. 


DOD  continued  restructuring, 
increasing  test  resources  and 
lowering  the  production  rate. 
Independent  review  teams 
evaluated  aircraft  and  engine 
manufacturing  processes.  Cost 
increases  later  resulted  in  a 
Nunn-McCurdy  breach.  Military 
services  are  currently  reviewing 
capability  requirements  as  we 
recommended. 


DOD  concurred  with  all  three  of 
the  recommendations.  DOD  lifted 
STOVL  probation  citing  improved 
performance.  Subsequently,  DOD 
further  reduced  procurement 
quantities,  decreasing  funding 
requirements  through  2016.  The 
initial  independent  software 
assessment  began  and  ongoing 
reviews  are  planned  to  continue 
through  2012. 
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Appendix  I:  Prior  GAO  Reports  and  DOD 
Responses 


GAO  report 

Est.  dev.  Costs  dev. 
Length 

Aircraft  unit  cost 

Key  program  event 

Primary  GAO  message 

DOD  response  and  actions 

2012 

GAO- 12-437 

$55.2  Billion 

1 8  years 
$137  Million 

The  program  established 
a  new  acquisition 
program  baseline  and 
approved  the 
continuation  of  system 
development,  increasing 
costs  for  development 
and  procurements  and 
extending  the  period  of 
planned  procurements  by 
2  years. 

Extensive  restructuring 
places  the  program  on  a 
more  achievable  course. 

Most  of  the  program’s 
instability  continues  to  be 
concurrency  of  development, 
test,  and  production.  We 
recommend  the  Cost 
Assessment  Program 
Evaluation  office  conduct  an 
analysis  on  the  impact  of 
lower  annual  funding  levels; 
JSF  program  office  conducts 
an  assessment  of  the  supply 
chain  and  transportation 
network. 

DOD  partially  concurred  with 
conducting  an  analysis  on  the 
impact  of  lower  annual  funding 
levels  and  concurred  with 
assessing  the  supply  chain  and 
transportation  network. 

2013 

GAO- 13-309 

$55.2  Billion 

1 8  years 
$137  Million 

The  program  continued 
to  move  forward  following 
a  new  acquisition 
program  baseline  in 

2012.  In  doing  so,  the 
program  incorporated 
positive  and  more 
realistic  restructuring 

The  program  is  moving  in  the 
right  direction  but  must  fully 
validate  design  and 
operational  performance  and 
at  the  same  time  make  the 
system  affordable.  We  did 
not  make  recommendations 
to  DOD. 

DOD  agreed  with  GAO’s 
observations. 

actions  taken  since  2010 
including  more  time  and 
funding  for  development 
and  deferred 
procurement  of  more 
than  400  aircraft  to  future 
years 


Source:  GAO 
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Appendix  II:  Scope  and  Methodology 


To  assess  the  program’s  ongoing  development  and  testing  we  reviewed 
the  status  of  software  development  and  integration  and  contractor 
management  improvement  initiatives.  We  also  interviewed  officials  from 
the  program  office,  prime  contractor,  and  the  Defense  Contract 
Management  Agency  (DCMA)  to  discuss  current  development  status  and 
software  releases.  In  addition,  we  compared  management  objectives  to 
progress  made  on  these  objectives  during  the  year.  We  obtained  and 
analyzed  data  on  flights  and  test  points,  both  planned  and  accomplished 
during  2013.  We  compared  test  progress  against  the  total  program  plans 
to  complete.  We  also  reviewed  the  Director,  Operational  Test  and 
Evaluation’s  annual  F-35  assessment.  In  addition,  we  interviewed  officials 
from  the  F-35  program  office  and  aircraft  prime  contractor  to  discuss 
development  test  plans  and  achievements.  We  also  collected  information 
from  the  program  office,  prime  contractor  and  Department  of  Defense 
(DOD)  test  pilots  regarding  the  program’s  technical  risks  including  the 
helmet  mounted  display,  autonomic  logistics  information  system,  carrier 
arresting  hook,  and  structural  durability. 

To  assess  the  program’s  funding  and  long-term  affordability,  we  reviewed 
financial  management  reports,  annual  Selected  Acquisition  Reports,  and 
monthly  status  reports  available  as  of  December  2013.  In  addition,  we 
reviewed  total  program  funding  requirements  from  the  Defense 
Acquisition  Executive  Summary.  We  used  this  data  to  project  annual 
funding  requirements  through  the  expected  end  of  the  F-35  acquisition  in 
2037.  We  also  analyzed  the  fiscal  year  2014  President’s  Budget  data  to 
identify  the  current  status  of  unit  costs  for  each  variant,  and  the 
differences  in  this  cost  since  2012.  We  reviewed  the  Office  of  the 
Secretary  of  Defense’s  F-35  Joint  Strike  Fighter  Concurrency  Quick  Look 
Review,  and  discussed  and  analyzed  reported  concurrency  costs  with  the 
prime  contractor  and  program  office.  We  obtained  and  discussed  the  life- 
cycle  operating  and  support  cost  through  the  program’s  Selected 
Acquisition  Report  and  projections  made  by  the  Cost  Analysis  and 
Program  Evaluation  (CAPE)  office.  We  identified  changes  in  cost  and 
interviewed  officials  from  the  program  office  prime  contractor,  Naval  Air 
Systems  Command,  and  the  CAPE  office  regarding  reasons  for  these 
changes.  We  also  discussed  future  plans  of  the  DOD  and  prime 
contractor  to  try  and  reduce  life-cycle  sustainment  costs  with  officials  from 
the  prime  contractor,  program  office,  and  CAPE.  We  analyzed  reliability 
data  and  discussed  these  issues  with  program  and  prime  contractor 
officials. 

To  assess  manufacturing  progress  we  obtained  and  analyzed  data 
related  to  aircraft  delivery  rates  and  work  performance  data  through  the 
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end  of  calendar  year  2013.  This  data  was  compared  to  program 
objectives  identified  in  these  areas  and  used  to  identify  trends.  We 
reviewed  data  and  briefings  provided  by  the  program  office,  prime 
contractor,  and  DCMA  in  order  to  identify  issues  in  manufacturing 
processes.  We  discussed  reasons  for  delivery  delays  and  plans  for 
improvement  with  the  prime  contractor.  We  also  toured  the  prime 
contractor’s  manufacturing  facility  in  Fort  Worth,  Texas  and  collected  and 
analyzed  data  related  to  aircraft  quality  through  December  2013.  We 
reviewed  and  discussed  information  on  the  prime  contractor’s  global 
supply  chain  including  their  management  processes  for  oversight. 

We  assessed  the  reliability  of  DOD  and  contractor  data  by  reviewing 
existing  information  about  the  data,  and  interviewing  agency  officials 
knowledgeable  about  the  data.  We  determined  that  the  data  were 
sufficiently  reliable  for  the  purposes  of  this  report.  We  conducted  this 
performance  audit  from  June  2013  to  March  2014  in  accordance  with 
generally  accepted  government  auditing  standards.  Those  standards 
require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on 
our  audit  objectives. 


Page  31 


GAO-14-322  F-35  Joint  Strike  Fighter 


Appendix  III:  Comments  from  the 
Department  of  Defense 


ASSISTANT  SECRETARY  OF  DEFENSE 
30 1 S  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3015 

031 914 

ACQUISITION 


Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report, 
GAO-14-322,  “F-35  JOINT  STRIKE  FIGHTER:  Problems  Completing  Software  Testing  May 
Hinder  Delivery  of  Expected  Warfighting  Capabilities,”  dated  February  28, 2014  (GAO  Code 
121 146).  Detailed  comments  on  the  report  are  enclosed. 

The  DoD  concurs  with  your  recommendation  and  we  appreciate  the  opportunity  to 
comment  on  the  draft  report.  My  point  of  contact  for  this  effort  is  Colonel  Todd  Levine, 
703-697-2573,  todd.j.levine.mil@mail.mil. 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  FEBRUARY  28, 2014 
GAO-14-322  (GAO  CODE  121146) 

“F-35  JOINT  STRIKE  FIGHTER:  PROBLEMS  COMPLETING  SOFTWARE  TESTING 
MAY  HINDER  DELIVERY  OF  EXPECTED  WARFIGHTING  CAPABILTIES” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION:  The  GAO  recommends  that  the  Secretary  of  Defense  conduct  an 
assessment  of  the  specific  capabilities  that  realistically  can  be  delivered  and  those  that  will  not 
likely  be  delivered  to  each  of  the  services  by  their  established  initial  operating  capability  dates. 
The  results  of  this  assessment  should  be  shared  with  Congress  and  the  military  services  as  soon 
as  possible  but  no  later  than  July  201 5. 

DoD  RESPONSE:  Concur.  The  Department  agrees  that  there  is  value  in  assessing  the  risk  of 
delivering  required  capabilities  within  the  stated  initial  operational  capability  windows  for  each 
military  service  in  order  to  make  timely  decisions  with  respect  to  future  investment  and  force 
structure.  The  Department  will  ensure  that  in  the  process  of  its  periodic  reviews  and  reports,  this 
assessment  is  conducted  and  shared  with  the  military  services  and  Congress. 
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